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ABSTRACT 

Decomposition of large problems into a hierarchic pyramid of subproblems was proposed in the literature 
as a means for optimization of engineering systems too large for “all-in-one'’ optimization This decompo- 
sition was established heuristically. The paper shows that the dynamic programing (DP) method due to 
Bellman when augmented with an optimum sensitivity analysis provides a mathematical basis for the above 
decomposition, and overcomes the “curse of dimensionality” that limited the original formulation of DP. 
Numerical examples are cited. 


INTRODUCTION 

Engineering optimization problems, e.g., maximization of aircraft performance, are usually computation- 
ally expensive and depend on the correct simulation of interaction of many parts and disciplines. Therefore 
these problems are natural candidates for optimization by decomposition that converts one large problem 
into a set of coordinated smaller problems. This paper shows that the Unear decomposition method whose 
appUcations recently appeared in the Uterature, may be viewed as an extension and generalization of the well- 
established algorithm of Dynamic Programing due to Bellman (ref. 1). In particular, this method alleviates 
to a large extent the algorithm limitation known as “the curse of dimensionality”. 

DYNAMIC PROGRAMING AS OPTIMIZATION BY DECOMPOSITION 

Dynamic Programing, originally due to Bellman (ref. 1), is perhaps the best mathematically established 
method for optimization by decomposition. Briefly summarized, the method applies to a system that may 
be represented by a train of the “black boxes" as in figure 1 (caUed the stages in ref. 1). The black boxes 
are mathematical models of the physical parts or conceptual aspects (engineering disciplines) of a large 
problem, and they are unified by a flow of information from the n-th box through the intermediate boxes 
ending with the 1-st box. Ai i-th box receives an input vector S,+l from its predecessor, an input vector of 
design variables .Y, and out puts a vector S; that becomes an input to the successor box >-l. The ;-th box 
also outputs a quantity Ri interpreted as a component of the objective function of the entire system. Each 
{S,} must be reducible to a function of a single variable, .s;, S t = /(.?,), and, because S, - /(S;+i. A,) and 
Ri = /(S;+i, -Y t ), it foUows that S, = f{s{+ \,Xi), Ri = /(s,+t,A;), and s; = /(•S'+l.X)- 

The problem of finding a set of vectors .Y„ i = 1 . . . n, that minimizes the sum of R t ,i = 1 . • ■ n. is 
solved by starting with the 1-st box at the end of the train. The variable * 2 governing the input o 2 is 
assumed to vary within an interval of interest. Several values of s 2 distributed over that interval are set and 
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an optimization problem, constrained or unconstrained, is solved to find {ATi } so as to minimize R\ at each 
value. That solution yields {ATi} 0 pt = f{s 2) and Rim^ = f(s 2), either in a discrete (a look-up table) form 
or in a continuous form interpolated between the $\ values assumed above, dependent ly on the nature of the 
problem. 

Moving up to the 2-nd box, one seeks for each of the several values of .93 in an interval of interest an 
optimal {A^} (denoted {A*2} 0 pt) that minimizes the sum R2 + R imin* One must consider that .*2 = -Y2) 

and that for each value of *2, there are {-Yi} 0 pt and Rimin already known from the optimizations that have 
been executed for box 1. This operation generates the values of {A^Jopt = / (^3) and (J?2 4- Rimin)niin- 

The procedure continues recursively from box i to box i-f-1, carrying forward {A*j} 0 pt, and (Rj 4- (Rj-x 4- 
(Rj- 2 . . . + (i?2 + Rimin) min) min) min • • .)min, j - 1 • * • h through the initial box in the train, i = n, whereby 
the minimum sum of all R^’s and a complete set of {A^} 0 pt’s gets established. The procedure rests on the 
fundamental principle formulated by Bellman which asserts that the set of {A” j } , j = 1 . . . n., is optimal when 
its subset for j = 1 . . . i taken for any i minimizes the sum of Rj, j = 1 . . . i, for S t - + 1 input given from the 
remainder of the train. 

The procedure computational cost heavily depends on the aforementioned assumptions of 5,* = f(s t ) 
where s, is a single variable. Indeed, if .?,■ were a vector of m elements, R^min would grow from a line plot 
into a hypersurface in m dimensions. Assuming a quadratic representation of that hypersurface (the lowest 
order nonlinear approximation), the n um ber of discrete points at which optimizations would have to be 
performed would grow proportional to the square of m, thus quickly destroying advantages of the procedure 
as a computational cost saver. Bellman called this the “curse of dimensionality” and regarded it as a barrier 
limiting applicability of the method. 

OVERCOMING THE CURSE OF DIMENSIONALITY 

Optimum sensitivity analysis formulated in ref. 2 provides a means for generalization of the above 
procedure to include defined as a vector of m elements. The optimum sensitivity analysis algorithm 
yields derivatives of the optimal {A } and R with respect to the parameters of the optimization problem 
(unconstrained or constrained). Taking box 1 as an example, {52} may now be defined as{52} = /({^2}), 
where {32} is a vector of elements 32 jt, k = 1 . . . m. For {^2} given, one may find {A'i} 0 pt and m i n i m u m of 
R\ and their derivatives with respect to each ^2^, regarded as an optimization parameter. Using the notation 
D(X\ y S2 Ai) and D(R\ 1 32 k) for these derivatives, the linear part of the Taylor series enables one to express 
{A*i}opt and Rimin as continuous, albeit approximate, functions of {$2}: 

{ATi}o P t = f({s 2}) = ({A'i} op t), + [D{Xu* 2 k)]M* 2 } (1) 

Rimin =/({<2» - (Rlmin)o+ {D(R U 3 2k )}' *{* 2 } (2) 

The Bellman’s Dynamic Programing procedure may now be executed using the above approximations 
in place of {ATi} op t — /(a 2) and Rimin = /(a 2), otherwise the procedure remains unchanged. The 
new component in the modified procedure is the optimum sensitivity analysis to be executed after each 
optimization involving boxes 1, (2 + 1), (34-2 + 1 ),... n, recursively. Because the linear relationships, eqs. 1 
and 2, introduce errors whose control requires move limi ts on design variables in each optimization, the entire 
procedure has to be repeated p times until satisfactory convergence is attained. In this case, the number p 
depends on the nonlinearities of the problem at hand. Consequently, because there is only one optimization 
in each box in one pass, the number of optimizations required to converge the procedure is pn. This is in 
contrast to nmm 1 which is necessary for the original procedure. The curse of dimensionality with respect 
to m is removed. The ratio pn/nmm = p/mm tends to be very small for large m and renders the modified 
procedure usable where the original one would be prohibitively expensive. 

However, unlik e the original procedure, the modified procedure relies on the continuity of the approxi- 
mation function in eqs. 1 and 2; hence, it cannot accommodate discrete design variables. 
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HIERARCHIC DECOMPOSITION 

Further generalization of the modified procedure is possible if the boxes in the train may be partitioned 
internally as shown in figure 2. This figure shows the boxes split internally into smaller ones. In this 
scheme, the train of boxes that was horizontal in figure 1 is depicted vertically to form a pyramid whose 
levels correspond to the boxes in figure 1. A typical level is populated by several boxes that formed a single 
box in figure 1. The pyramidal arrangement emphasizes the hierarchic dependence of the boxes in level i 
(“children”) on the information transmitted from a box located at the level above j > i ( “parent ), with 
the underlying assumption that the boxes at the same level (“siblings”) do not exchange information with 
each other directly. Similar tc the system shown in figure 1, the behavior information from each box flows in 
figure 2 from the parent to the children, or from the top level n down to level 1. The optimization information 
from each box flows in the opposite direction. This information includes the optimum sensitivity derivatives 
that enable optimization in each parent box to be performed taking into account the effect of its {A'} on the 
optimization results in all boxes descendent from that parent. 

Thus, the above decomposition scheme first developed heuristically in ref. 3 is shown to be a generalization 
of the Bellman’s Dynamic Programing. The scheme became known as hierarchic, linear decomposition. 


APPLICATION EXAMPLES 

Since its introduction in ref. 3, optimization by hierarchic linear decomposition has been demonstrated 
to be useful in several applications. For example, in ref. 4 it was used to develop structural optimization by 
substructuring. This case is illustrated by a portal framework (figure 3a) shown decomposed in figure 3b. The 
procedure histogram in figure 3c exhibits satisfactory convergence characteristics. Analytical information 
flowing down the pyramid consisted of the internal forces and cross-section stiffness properties as parameters 
of optimization. Optimal cross-sectional dimensions, minimal values of the cumulative constraints and their 
sensitivity derivatives with respect to the above parameters were transmitted in the opposite direction. 

An example of the procedure application to a multidisciplinary problem of optimization of a transport 
aircraft for performance under constraints drawn from major contributing disciplines was described in ref. o. 
The aircraft, its decomposition scheme, and a histogram of the optimization procedure are shown in figure 4 
a, b, and c, respectively. The case featured over 1000 design variables and constraints and demonstrated 
a mathematical link from tae design detail (e.g., wing panel cross-sectional dimensions) to the system 
performance (e.g., the mission fuel). The procedure convergence was smooth and rapid as seen m figure 4c. 


CONCLUSIONS 

It is shown that the Bellman’s method for decomposition of large optimization problems known as 
Dynamic Programing may be generalized to encompass the cases when the information transmitted between 
the parts of the system is a function of many variables. The key component of the modified procedure is the 
derivatives of optimum with respect to the optimization parameters. Application examples illustrate and 
verify the procedure. 
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